6.9. Operator Right

#6.9. Operator Right | 来源: 网络整理| 查看: 265

6.9.11. Use Case - 0x04露

This is our function library.

Transformation functions (non-reducing) - takes one argument and returns one value:

>>> def increment(x): ... return x + 1 >>> >>> def decrement(x): ... return x - 1 >>> >>> def square(x): ... return x ** 2 >>> >>> def cube(x): ... return x ** 3

Reducing functions - takes two arguments returns one value:

>>> def add(x, y): ... return x + y >>> >>> def sub(x, y): ... return x - y >>> >>> def mul(x, y): ... return x * x

We have data to compute:

>>> data = [ ... [1, 2, 3], ... [4, 5, 6], ... [7, 8, 9], ... ]

On this data, we want to apply the following transformations:

>>> transformations = [increment, square, decrement, cube]

We need to create apply function, which takes data and apply the transformation:

>>> def apply(data, fn): ... return map(fn, data)

Let's do it parallel. We will create three independent workers. Each worker will get part of the data (one-third) and will apply all the transformation (map) to their data subset.

>>> workerA = reduce(apply, transformations, data[0]) # [27, 512, 3375] >>> workerB = reduce(apply, transformations, data[1]) # [13824, 42875, 110592] >>> workerC = reduce(apply, transformations, data[2]) # [250047, 512000, 970299]

Note, that all workers will produce generators (maps). We need to merge the results using reduce function, but before that we need to evaluate maps to lists.

>>> def merge(x, y): ... return list(x) + list(y) >>> merged = reduce(merge, [workerA, workerB, workerC]) >>> result = reduce(add, merged) >>> print(result) 1903551 >>> print(merged) [27, 512, 3375, 13824, 42875, 110592, 250047, 512000, 970299]

【本文地址】

公司简介

联系我们